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DETAILED ACTION 

Amendments and Remarks 

1 . Applicant's amendments and remarks dated 24 July 2007, have been entered and made of 
record. 

Response to Amendments and Remarks 

2. Applicant's arguments have been fully considered but they are not persuasive. In response 
to applicant's argument that Marcelpoil (7,056,236) and Kotera (4,090,243) are nonanalogous 
art, it has been held that a prior art reference must either be in the field of applicant's endeavor 
or, if not, then be reasonably pertinent to the particular problem with which the applicant was 
concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re 
Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). 

In this case, Applicant argues not that Marcelpoil and Kotera are not analogous to the 
claimed invention, but rather that Marcelpoil and Kotera cannot be combined because the field of 
endeavor of Kotera is not analogous to that of Marcelpoil. Applicant points out various 
differences between the disclosures of Marcelpoil and Kotera (see Remarks, p. 8). 
Notwithstanding these differences, however, Examiner considers the references to be sufficiently 
closely related for the purposes of § 103 analysis. 

Fundamentally, Marcelpoil and Kotera are related to the broad field of image analysis, 
and more particularly, to the narrower problem of determining the constituent color(s) contained 
in a sample under examination, in accordance with the claimed invention of "quantifying a color 
in a sample." In Marcelpoil, the samples under examination are stained or dyed biological cells, 
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whereas in Kotera, the samples are painted color prints prepared by artisans. Although the 
examination of stained cells and the examination of painted prints per se cannot be considered 
related endeavors, both Marcelpoil and Kotera are directed to determining colors present in those 
samples, and the methods of analysis that each employ — such as defining a matrix based on 
color values and utilizing an inverse of that matrix based on control measurements as a 
conversion matrix to calculate the amount of color in the sample (e.g., Kotera, column 4/32 
through column 5/66) — are very similar. Accordingly, the disclosures of Marcelpoil and Kotera 
are considered related at least in the sense that both are directed to analogous problems in the 
field of image analysis, and both seek solutions to those problems in similar manners. 

Claim Objections 

3. In view of Applicant's amendments to claims 7 and 16, the previous objections thereto 
have been withdrawn. 



Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 1-17 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 7,065,236 by Marcelpoil et al. ("Marcelpoil") in view of U.S. Patent 4,090,243 by Kotera 



et al. ("Kotera"). 
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Regarding claims 1 and 10, Marcelpoil discloses a method/program for quantifying color 
in a sample comprising multiple colors, the method comprising: 

measuring a color channel value in a plurality of pixels from a control sample comprising 
a single color of interest (column 8/14-23: camera 300 captures a color image of a sample 500 - 
the image having red, green, and blue color channel values); 

defining a vector for the control sample, wherein the vector comprises a color channel 
value present in the control (e.g. the optical density vector OD, given by equations 3-5 or 6-8 in 
column 1 1 , defines a vector comprising the measured optical densities for the red, green, and 
blue color channels); 

defining a matrix comprising each of the values of each of the color channels (i.e. the 
matrix formed by the equations associated with the OD vector is defined by equations 21 and 22 
in column 14, in order to determine the dye concentrations C based on the known optical 
densities OD and absorption coefficients e - see column 14/1-6); 

defining a conversion matrix comprising the inverse of the matrix based upon the control 
measurements (i.e. the conversion matrix denoted by equation 23 in column 14 is defined based 
upon the measured control optical densities); 

measuring color channel values in an image of an experimental sample comprising a 
plurality of colors of interest, each of the pixels comprising a plurality of color channels (column 
16/9-14: an experimental sample having the same dyes uses in the calibration process for 
determining the conversion matrix is imaged in the same manner as the control sample); and 

calculating the amount of a color in the experimental sample by converting the channel 
values in the experimental sample using the conversion matrix (column 16/14-31 : the amount of 
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color, or concentrations of the dyes, in the experimental sample is determined using the 
conversion matrix). 

However, Marcelpoil seems to only utilize a single control sample and does not appear to 
disclose or suggest using a "plurality of control samples," as claimed. Accordingly, Marcelpoil 
does not disclose defining the vector or the matrix on the basis of an "average" of color channel 
values for a "plurality of control samples." 

Kotera discloses a system (figures 1 A and IB) for characterizing the colors of a color 
sample that is very similar to that of Marcelpoil and involves the same concepts of deriving an 
inverse matrix of mean color intensity values (column 5/1-35) and using the inverse matrix to 
ascertain the colors of an experimental sample (column 5/58-66). In particular, Kotera teaches 
that control color prints CI . . . Cn, as shown in figure 2, are imaged, and the measured colors and 
used to determine the conversion matrix. Kotera teaches that each of these control color prints 
contain a plurality of "elemental areas," which essentially correspond to sub-areas within the 
larger print area, and the elemental areas are each "microscopically" imaged to generate 
representative intensity signals thereof (column 1/59 through column 2/3). The mean intensity 
value ix of each spectral (i.e. color) component for each control color print corresponds to the 
representative intensity signal of a per-unit area (column 2/18-27). The set of mean values ja, in 
conjunction with the set of representative intensity signals, is utilized to obtain the conversion 
matrix that is used to ascertain the color Q of a given experimental sample, expressed as the 
probability P(Q) that the ascertained color corresponds to the i-th control print (see column 2/22- 
39 and column 3/57 et seq.). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Marcelpoil by Kotera to achieve the claimed invention by measuring a plurality of 
control samples and defining the vector and the matrix on the basis of an average of each color 
channel present in the control samples, since Kotera shows that it was conventional to 
microscopically scan a control color print on the basis of a plurality of "elemental" sample areas 
and utilize the average of those sample areas as a basis for deriving a conversion matrix, as 
explained above. Such a modification would achieve substantially the same results as achieved 
by Marcelpoil since Marcelpoil's control samples are analyzed in a region where two or three 
dyes are mixed to produce an area of uniform color. 

Regarding claims 2 and 11, Marcelpoil discloses the color channels comprise red, green, 
and blue (see figure 1). 

Regarding claims 3 and 12, the combination of Marcelpoil and Kotera discloses each 
control is stained with a single staining reagent to generate a color of interest (column 9/13-17 
and column 10/63 et seq. of Marcelpoil: the control is stained with a plurality of staining 
reagents, including a single marker dye that is used to generate a color of interest). 

Regarding claims 4 and 13, Marcelpoil discloses that the experimental sample is stained 
with a plurality of stains to generate a plurality of colors of interest (column 10/63 et seq.: the 
sample is stained with e.g. a marker dye and a counters tain). 

Regarding claims 5 and 14, Marcelpoil discloses that the number of stains in a 
experimental sample are less than or equal to the number of color channels (column 16/14-31 : 
concentrations of 3 dyes are determined - with there being 3 color channels). 
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Regarding claims 6 and 15, Marcelpoil suggests that an image of the experimental sample 
can be displayed as a monochrome image (see e.g. equation 24, column 16, which quantifies the 
black and white pixel intensities for the experimental sample image). 

Regarding claims 7 and 1 6, Marcelpoil does not expressly disclose setting all but one of 
the color channel values to zero in order to determine the amount of a single color in the 
experimental sample, as claimed, however, such a limitation would have been exceedingly well- 
known and obvious to those skilled in the art in view of the fact that each of the color channels 
for an RGB image independently quantify the amount of a single color - red, green, or blue - 
present in an image, and e.g. the values of red and green channels have no bearing on how much 
"blue" is exhibited by an image of the sample. 

Regarding claims 8 and 17, Marcelpoil discloses rendering a digital display of the 
experimental sample (i.e. displayed on the computer screen, as shown in figures 1 and 2). 

Regarding claim 9, the combination of Marcelpoil and Kotera teaches the computer 
implemented method of claim 1 (i.e. both Marcelpoil's and Kotera's methods are implemented 
via a computer). 

Regarding claim 22, Marcelpoil discloses that the computer-implemented method can be 
automated (see column 20/2-14). 

6. Claim 18 is rejected under 35 U.S.C, 103(a) as being unpatentable over U.S. Patent 
7,065,236 by Marcelpoil et al. ("Marcelpoil") in view of U.S. Patent 4,090,243 by Kotera et al. 
("Kotera") and U.S. Patent Application Publication 2004/01 14227 by Henderson et al. 
("Henderson"). 
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Regarding claim 18, Marcelpoil discloses a machine vision system (figures 1 and 2) for 
automated analysis of a biological sample on a slide comprising: 

a system processor (i.e. computer 350 includes a processor); 
a computer program on computer readable medium (column 20/2-14), the computer 
program comprising an image algorithm comprising instructions to cause the computer to: 

measure a color channel value in a plurality of pixels from a control sample 
comprising a single color of interest (column 8/14-23: camera 300 captures a color image of a 
sample 500 - the image having red, green, and blue color channel values); 

define a vector for the control sample, wherein the vector comprises a color 
channel value present in the control (e.g. the optical density vector OD, given by equations 3-5 
or 6-8 in column 11, defines a vector comprising the measured optical densities for the red, 
green, and blue color channels); 

define a matrix comprising each of the values of each of the color channels (i.e. 
the matrix formed by the equations associated with the OD vector is defined by equations 21 and 
22 in column 14, in order to determine the dye concentrations C based on the known optical 
densities OD and absorption coefficients 8 - see column 14/1-6); 

define a conversion matrix comprising the inverse of the matrix based upon the 
control measurements (i.e. the conversion matrix denoted by equation 23 in column 14 is defined 
based upon the measured control optical densities); 

measure color channel values in an image of an experimental sample comprising a 
plurality of colors of interest, each of the pixels comprising a plurality of color channels (column 



Application/Control Number: 10/822,561 Page 9 

Art Unit: 2624 

16/9-14: an experimental sample having the same dyes uses in the calibration process for 
determining the conversion matrix is imaged in the same manner as the control sample); and 
calculate the amount of a color in the experimental sample by converting the 
channel values in the experimental sample using the conversion matrix (column 16/14-31 : the 
amount of color, or concentrations of the dyes, in the experimental sample is determined using 
the conversion matrix); and 

output the amount of color in the experimental sample (column 17/1-19); 
a monitor in operable communication with the computer (as shown in figure 1); 
an input device in connection with the computer (e.g. keyboard or mouse shown in figure 

2); 

an optical imaging system (video microscopy system 100) in operable communication 
with the computer, comprising: 

a movable stage (column 18/59-63); 

an identification member (column 17/28-45: identification marks produced by an 

operator); 

an optical sensing member (camera 300) in optical communication with the stage 
configured to acquire an image at a location on a slide and in electrical communication with the 
processor; 

a storage member for storing the location of a candidate object or area of interest 
(column 17/20-64 and column 19/28-46: the memory of the computer 350 is used to store images 
containing markings that indicate the locations of areas of interest); and 

a storage device for storing each image (column 19/22-32). 
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However, Marcelpoil seems to only utilize a single control sample and does not appear to 
disclose or suggest using a "plurality of control samples," as claimed. Accordingly, Marcelpoil 
does not disclose defining the vector or the matrix on the basis of an "average" of color channel 
values for a "plurality of control samples." 

Kotera discloses a system (figures 1 A and IB) for characterizing the colors of a color 
sample that is very similar to that of Marcelpoil and involves the same concepts of deriving an 
inverse matrix of mean color intensity values (column 5/1-35) and using the inverse matrix to 
ascertain the colors of an experimental sample (column 5/58-66). In particular, Kotera teaches 
that control color prints CI . . . Cn, as shown in figure 2, are imaged, and the measured colors and 
used to determine the conversion matrix. Kotera teaches that each of these control color prints 
contain a plurality of "elemental areas," which essentially correspond to sub-areas within the 
larger print area, and the elemental areas are each "microscopically" imaged to generate 
representative intensity signals thereof (column 1/59 through column 2/3). The mean intensity 
value (J, of each spectral (i.e. color) component for each control color print corresponds to the 
representative intensity signal of a per-unit area (column 2/18-27). The set of mean values ja, in 
conjunction with the set of representative intensity signals, is utilized to obtain the conversion 
matrix that is used to ascertain the color C, of a given experimental sample, expressed as the 
probability P(Cj) that the ascertained color corresponds to the i-th control print (see column 2/22- 
39 and column 3/57 et seq.). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Marcelpoil by Kotera to measure a plurality of control samples and define the vector 
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and the matrix on the basis of an average of each color channel present in the control samples, as 
claimed, since Kotera shows that it was conventional to microscopically scan a control color 
print on the basis of a plurality of "elemental" sample areas and utilize the average of those 
sample areas as a basis for deriving a conversion matrix, as explained above. Such a 
modification would achieve substantially the same results as achieved by Marcelpoil since 
Marcelpoil's control samples are analyzed in a region where two or three dyes are mixed to 
produce an area of uniform color. 

In addition, Marcelpoil discloses that the microscope may include one or more robotic 
components (column 1 8/59-63) but does not appear to disclose an automated loading and 
unloading member for loading and unloading of a slide, as claimed. 

Henderson discloses an automated slide loader for use with a microscope. In particular, 
Henderson teaches that it is advantageous to provide an apparatus that automatically loads and 
unloads slides to and from a microscope. It would have been obvious to one of ordinary skill in 
the art at the time of the invention to modify Marcelpoil and Kotera to achieve the claimed 
invention by including an automatic slide loader/unloader, as claimed, since automating a 
manual procedure has been judicially recognized as per se obvious. See In re Venner, 262 F.2d 
91, 95, 120 USPQ 193, 194 (CCPA 1958). 

Conclusion 

7. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 . 1 36(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (571) 272-7423. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Brian 
Werner, can be reached on (571) 272-7401 . The fax phone number for the organization where 
this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you haye questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. Any inquiry 
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of a general nature or relating to the status of this application or proceeding can also be directed 
to the TC 2600 Customer Service Office whose telephone number is (571) 272-2600. 



Colin M. LaRose 
Group Art Unit 2624 
9 October 2007 



